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Introduction 

Understanding how students learn engineering design 
concepts and the subsequent instructional interventions that are 
directed to improve their performance is also contingent on 
understanding how experts in the various engineering 
disciplines solve engineering problems. Naturally, a part of the 
engineering and technology educators’ research agenda is 
aimed at gaining a better insight of how student and expert 
engineers solve specific engineering problems. This hopefully 
would lead to a larger body of knowledge that is accessible for 
administrators and teachers to make infonned decisions about 
the teaching of engineering design concepts. 

Over the past two decades a steady proliferation of 
studies in engineering problem solving have focused on the 
differences between expert and novice designers (Cross, 2002, 
2004), design reasoning and thinking (Goldschmid & Weil, 
1998), creativity and design (Christiaans & Venselaar, 2005; 
Dost & Cross, 2001) and the design processes and strategies of 
engineering students (Atman & Bursic, 1998; Cardella, Atman, 


Raymond Dixon is a Research Associate at the Center for Mathematics, Science and 
Technology at Illinois State University. He can be reached at 
rdixonenator@ email .com . 


Number 1 
36 


Volume 48 


2011 




Core Thinking Skills And Cognitive Strategy 


37 


Turns & Adams, 2008; Merrill, Custer, Daugherty, Westrick, 
& Zeng, 2007). In a recent study by Merrill et al. (2007) three 
core engineering design problem solving concepts were 
identified as important to teach at the high school level— 
constraint, optimization, and predictive analysis (COPA). 
These concepts reflect the processes that the professional 
designer uses in the real world to solve design problems. The 
ill-structured nature of engineering design problem solving also 
demands the use of high level thinking skills such as analyzing 
and generating skills (Ullman, 2003; Atman & Bursic, 1998). 
These skills are inextricably linked to the strategies used by 
experts and novices as they work with the constraints of a 
design problem, find an optimal solution, and use various 
analytical procedures. 

When engineering design is examined from the 
perspective of the problem space and solution space (Dorst & 
Cross, 2001), these two mental spaces represents spaces of 
association between core thinking skills and core engineering 
design processes such as COPA. The problem space includes 
activities such as defining the problem, identifying constraints, 
specifying evaluation criteria, and gathering information about 
various solutions. The generation of solutions and the 
execution of problem solving strategies define the solution 
space. Specifically, this includes activities such as making 
decisions about various possible solutions, performing analysis, 
optimizing the selected solution, and determining 
specifications. 


Purpose of the Study 

The purpose of this study was to use verbal protocol 
analysis to explore qualitatively how the analyzing and 
generating thinking skills of a student and an expert engineer 
differ, and to determine how these core thinking skills 
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influences their overall cognitive strategy in the problem and 
solution spaces as they solved a common engineering design 
problem. 

Research Questions 

The following research questions guided this study: 

1. How do the analyzing skills of a student and an expert 
engineer differ when using core engineering design 
concepts in solving a design problem? 

2. How do the generating skills of a student and an expert 
engineer differ when using core engineering design 
concepts in solving a design problem? 

3. What are the dominant cognitive strategies used by the 
engineering student and the expert engineer? 

Thinking Skills, Design Concepts, and Design Strategies 
Core Thinking Skills 

Researchers identified several core thinking skills that 
are used by individuals in cognitive processing and for creative 
and critical thinking. These skills are valued by educators as 
important for learning and problem solving (Marzano et ah, 
1988). They are grouped into eight categories: focusing skills, 
information gathering skills, remembering skills, 
organizational skills, analyzing skills, generating skills, 
integrating skills and evaluating skills. For the purpose of this 
study, analyzing and generating thinking skills were examined 
because these skills constitute key cognitive activities of 
engineers when they solve design problems. 

As engineers solve problems, they generate various 
types of conceptual solutions and perform analyses using 
different mathematical strategies and heuristics (Ullman, 
2003). According to Marzano et al. (1998), “analyzing skills 
are used to clarify existing information by examining their 
parts and relationships” (p. 91). To analyze, one must be able 
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to identify attributes and components, identify relationships 
and patterns, and identify errors. 

In engineering design, the ability to identify attributes 
and components helps students to focus analytically on the 
structure of objects, their systems, and fonns. Identifying 
relationships and patterns allows students to recognize and 
articulate the interrelationship and the functionality of 
component parts. Several studies have examined the 
importance of identifying relationships and patterns. In a study 
conducted by Egan and Schwartz (1979), they found that the 
recall of circuit drawings by skilled technicians was 
remarkably similar to the recall of chess positions by expert 
chess players. They referred to this aspect of pattern 
relationship as chunking. In addition, they verified that experts 
are able to improve their memory and problem-solving 
capability by identifying conceptual relationships. This is in 
contrast to novices, who because of a lack of domain-specific 
experience arrange patterns according to positional 
relationships. 

Ball, Omerod, and Morley (2004) conducted think- 
aloud protocols of expert engineers with a minimum of 7 years 
of academic and commercial design experience, and novices 
who were master’s engineering students with limited design 
experience. Each participant received an identical brief that 
related to the design of an automated car-rental facility. This 
brief was designed “to be complex, multifaceted, and ill- 
defined in the traditional sense of a prototypical design 
problem but tractable enough to be tackled to a satisfactory 
level by designers with only a few years of design experience” 
(p. 502). They found that experts displayed greater evidence of 
analogical reasoning than do novices, irrespective of whether 
such analogizing is “schema-driven” or “case-driven.” 
Schema-driven analogizing involves “the recognition-primed 
application of abstract experiential knowledge that could afford 
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a design solution to a familiar problem type” while case-driven 
analogizing entails “the invocation of a concrete prior design 
problem whose solution elements could be mapped onto the 
current problem.” They also found that the expert designers 
showed more evidence of schema-driven analogizing than 
case-driven analogizing, while the novice designers showed 
more evidence of case-driven analogizing than schema-driven 
analogizing. In other words, expert designers are more 
proficient in recognizing design problems with similar 
underlying conceptual relationships than do students or novice 
engineers. 

Identifying errors, the third analyzing skill, involves 
detecting flaws that may exist in knowledge, logic, calculation, 
or procedure. This analysis also extends, where possible, to 
actions that identify the causes and make corrections where 
necessary (Marzano, et ah, 1988). It is often postulated that 
people use mental models or their own naive theories to help 
them understand how complex systems behave (Gentner 2002; 
Collins, 1985). These mental models and theories assist in the 
diagnosing of error. Because of their experience, experts have 
more sophisticated causal mental models and theories that are 
governed by concepts from several related domains (Kempton, 
1986) and so they are naturally more efficient at identifying 
errors in their design conceptualizations. 

Generating skills enables an individual to use prior 
knowledge to add information beyond what is given (Marzano 
et ah, 1988). The student therefore uses his/her knowledge of 
the sciences, technical drawing, and the function of 
mechanisms to generate or construct a new device or system. 
This cognitive process involves connecting new ideas with 
prior knowledge to build a coherent organization that houses 
both new and old knowledge structures, which represents the 
interpretation of a new situation. Generating skills involves 
inferring, predicting, and, elaborating. Inferring involves 
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deductive and inductive reasoning. Predicting is making a 
statement or expectation anticipating the outcomes of a 
situation based on prior knowledge of how things usually turn 
out. Elaborating involves adding details, explanations, 
examples, or other relevant information from prior knowledge 
to improve understanding. Elaborating can be complex and 
involves constructing mental models and analogies to 
understand the structural and functional features of objects and 
systems. 

According to Bedard and Chi (1992), experts are more 
efficient and superior in classifying problems according to 
relevant features. They are also efficient in their inference 
about additional aspects of the problem. Experts represent 
problems according to their conceptual features, and spend a 
considerable amount of time developing their representation by 
adding domain specific and general constraints. In contrast, 
novices’ representations are largely based on literal features 
and they may attempt to solve problems directly without 
properly defining them. The complex mental models that 
experts are able to generate make it easier for them to make 
accurate inferences and predictions, and elaborate about their 
solutions perfonnance and the functionality of the designed 
component. Cross (2004) indicated that experienced designers 
use more generative reasoning in contrast to less experienced 
designers who use more deductive reasoning. In addition, 
expert designers select features of the problem space to which 
they chose to attend (naming) and identify areas of the solution 
space which they chose to explore (framing). Some expert 
designers (architects) approach to problem solving was 
characterized by strong paradigms or guiding themes, while 
novices had weaker guiding themes. 
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Core Engineering Design Concepts 

Although engineers use various approaches and 
methods to solve design problems, three core engineering 
design problem solving concepts were identified as important 
to teach at the high school level—constraint, optimization, and 
predictive analysis (COPA) (Merrill et al., 2007). 

Various types of analytical methods and models are 
used in engineering design to predict the perfonnance of 
artifacts and systems. The applicability of an analysis method 
is dependent on the level of accuracy needed and the 
availability of sufficient methods. General analytical methods 
are less expensive and faster to implement than physical 
modeling methods (Ullman, 2003). Ullman gave an example 
of how the stiffness of a diving board can be determined by 
using a method from the strength of materials: 

.. .the board is assumed to be a cantilever beam made of 
one piece of material of constant prismatic cross 
section, and with known moment of inertia. Further, 
the load of a diver bouncing on the end of the board is 
estimated to be a constant point load. With this 
analysis, the important variable—the energy storage 
properties of the board, its deflection, and the maximum 
stress— can be estimated, (p. 264) 

Predictive analysis should be carried out in the planning 
environment and not the task environment because moves 
made in the planning environment can be easily undone, while 
the task environment actions cannot be reversed. 

In the initiation of a new design problem, the design 
requirements such as specification and conditions for function, 
effectively constrain the possible solutions to a subset of all 
possible product designs. As the design process continues, 
other constraints are added to further reduce the potential 
solutions to the problem, and potential solutions are continually 
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eliminated until there is only one final product design (Ulman, 
2003). Beyond the constraints of the original problem, 
constraints that are created during the design process come 
from the designer’s knowledge of mechanisms, devices, and 
systems, and also from the design decisions made during the 
designing process. Other sources of constraints are costs, 
economics, feasibility, time, material, and environmental 
implications. 

The purpose of optimization is to achieve the best 
design based on prioritized constraints and criteria. This 
includes maximizing factors such as productivity, strength, 
reliability, longevity, cost, efficiency, and utilization (Merrill et 
ah, 2007). The functional parameters of designed components 
can be converted to some mathematical formula to detennine 
the optimal functioning capability of the designed component. 

Cognitive Strategy 

Designers use different strategies to solve a design 
problem. For example, some designers may begin solving a 
problem by deciding whether the process should be one of 
design or redesign. Another group of designers may prioritize 
certain stakeholders and strategize their solutions around the 
high-priority stakeholders. Other designers may arrange their 
design assignment to be new and challenging in order to 
provoke a creative response (Dorst & Cross, 2001). 

In a protocol analysis of nine experienced designers 
solving a design problem, Kruger and Cross (2006) categorized 
the strategies used into four general approaches. These are: 

• Problem-driven design. The designer focuses closely 
on the problem at hand and uses only infonnation and 
knowledge that are strictly needed to solve the problem. 
The emphasis lies in defining the problem and finding a 
solution as soon as possible. 
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• Infomiation-driven design. The designer focuses on 
gathering information from external sources that have 
not yet been processed and develops a solution on the 
basis of this information. 

• Solution-driven design. The designer focuses on 
generating solutions and gathers only infonnation that 
is needed to further develop a solution. The emphasis 
lies on generating solutions, and little time is spent on 
defining the problem. 

• Knowledge-driven design. The designer focuses on 
using prior highly structured, individual knowledge and 
develops a solution on the basis of this knowledge. 
Only minimal necessary information from external 
sources is gathered. 

The Conceptual Framework 

Figure 1 represents the framework that was used in this 
study. Analyzing and generating skills are used by designers in 
varying degrees as both mental spaces (problem space and 
solution space) evolve and as they work with various 
constraints, and carry out predictive analysis and optimization. 
This is illustrated in the diagram by the two ellipses. The 
design strategies used by each engineering designer will also 
vary and may include one or a combination of problem, 
information, solution and knowledge driven strategies. 
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Figure 1. 

Conceptual framework 
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Verbal Protocol Analysis 

A protocol is a “description of activities ordered in 
time, in which a subject engages while performing a task” 
(Hayes, 1989, p.51). Verbal Protocol Analysis (VPA), also 
known as “think-aloud” protocols, are often collected during 
(concurrent protocols) and after (reflective or retrospective 
protocols) problem solving episodes, to obtain a record of the 
knowledge used by the problem solver, and the succession of 
mental states through which he or she passes while working on 
the problem (Proctor & Dutta, 1995). When conducting a 
verbal protocol, the participants are asked to say aloud 
everything they think, while performing the task, no matter 
how trivial it seems. The obvious benefits of this type of 
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analysis include the relative ease with which participants 
typically verbalize their thoughts, and the potential for insight 
into their cognitive processes. Once the verbal protocols are 
collected by audio and/or video, they are transcribed, 
segmented into codable units of subject statements, coded 
according to a coding scheme, and analyzed to answer specific 
research questions. 

Think-aloud protocol has been used extensively in 
reading and comprehension studies (Donndelinger, 2005). 
Atman and Bursic (1998) argued that concurrent report is a 
valid method that can be used to collect data about someone’s 
thinking process. However, some have expressed concern that 
think-aloud protocols may distort or interfere with the mental 
processes that we seek to observe (Proctor & Dutta, 1995). 
Others contend that when protocols are collected properly it 
does not distort or interfere with the participant’s thinking and 
performance, because information is being collected from the 
short term memory, while subjects are prompted to “keep 
talking” with minimal interference from the experimenter(see 
Christensen & Yasar, 2007; Ericsson & Simon, 1993). 

Verbal protocol analysis has also been used by several 
researchers in engineering design to understand the cognitive 
process of experts and novice designers (see Atman & Bursic, 
1998; Ball, Onnerod, & Morley, 2004; Christensen & Schunn, 
2007; Christiaan & Dorst, 1992; Cross, 2002; Dorst & Cross, 
2001). A more recent study by Cardella, Atman, Turns, and 
Adams (2008) investigated the changes in individual 
engineering students design process over their course and how 
these changes might prepare them to become global engineers. 
Verbal protocol analysis was used to gain insight of the design 
behavior of engineering students as well as faculty members. 
A total of 61 students from various engineering disciplines 
participated. Some of their findings revealed that the more 
experienced designers (seniors) tend to spend more time in 
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design activities such as evaluating design alternatives, making 
design decisions, and communicating design decisions. Senior 
engineering students had more complete design solutions. 
Their solutions also had additional mechanical and technical 
features. Finally, they found that differences in “the structure 
of the task may affect students’ use of ‘analytical skills’, their 
‘holistic, multidisciplinary thinking’, their tendency to ‘exhibit 
creativity’, the extent to which they exhibit ‘high ethical 
standards and a strong sense of professionalism’ and their use 
of ‘the principles of business management’” ( p. 257). 

Methodology 

Purposeful sampling was used with a multiple-case 
study design. According to Gall, Gall, and Borg (2007), in 
purposeful sampling, the goal is to select cases that are likely to 
be information rich with respect to the purposes of the study. 
In a multiple-case design, the unit of analysis is two or more 
individuals, or two or more instances of phenomena that are 
collectively studied in one case. For the purpose of this study 
two participants with significant differences in their years of 
experience were used—an experienced engineer and a 
engineering student. Verbal or “think aloud” protocols were 
collected from each participant as they solved a design problem 
within an artificial context. 

The Participants 

The novice was a senior mechanical engineering 
student who worked as a teaching assistant for a computer- 
aided design course. The expert had a Bachelor’s degree in 
mechanical engineering and over 28 years of work experience. 
This included working as a manufacturing engineer and a 
locomotive engineer. He is also recognized as an expert 
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builder of large size locomotive models. That located him well 
above the number of years that it normally takes one to achieve 
expertise. According to Simon and Chase (1973), experts 
needed about 10 years of intense involvement in domain- 
specific activities before they can reach international levels of 
performance. 


The Task 

Each participant was given an ill-defined mechanical 
engineering design problem to solve. The engineering design 
problem was taken from a mechanical design text and modified 
to make it more ill-defined and more challenging. The task 
was then sent for validation to two engineering instructors, 
each with over 25 years of faculty experience teaching 
mechanical engineering. Feedback provided by each of the 
instructors resulted in minor modifications of the design task 
(see Figure 2). 
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Figure 2 
The Design Task 


Designing Task 

Overview 

The objective of this engineering designing activity is to 
understand better the cognitive process of engineering 
designers as they solve a design problem. The process 
that will be used is a Verbal Protocol Analysis. This 
means that as you solve the problem you will be required 
to “think aloud” (say aloud) what you are thinking. If 
you stop speaking I will remind you to resume speaking a 
loud as you solve the problem. The designing challenge 
below should not take you more than 1 hour to complete. 
The information from his activity will provide deeper 
insight in how to teach engineering designing and 
develop engineering designing curricula for K12 
institutions. 

The Task 

Design a quick release hold - down device used for 
holding down work-piece in a wood or metal shop. The 
devise must be able to hold material up to 3 inches thick 
and have at least an 8 inch reach. It should have the 
ability to release the work piece quickly and should be 
easy to position and move to other work surfaces. The 
holding strength of the device should also be considered. 
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Procedure 

Concurrent verbal protocols were collected from each 
participant as they solved the design problem. Verbal protocol 
analysis requires participants to “think aloud” while solving a 
problem or performing a task (Atman & Bursic, 1998). The 
novice (engineering student) selected a computer lab on 
campus to solve the design task while the expert used a 
drafting office at his home. Both were given pencils and 
sketch pads, and both were allowed to use the drafting software 
on their computer. They were each given an hour to complete 
the problem and were prompted to continue to speak aloud 
what they are thinking whenever they became silent. Each 
“think aloud” session was recorded digitally. 

Data Analysis 

After each participant completed the design task, the 
audio recordings of their concurrent protocols were transcribed. 
The transcribed protocols were then segmented into thi nk - 
aloud utterances, divided into sentences, and coded. The 
quality of the sketches was not evaluated since the objective of 
the study was to examine the mental processes of the 
engineering student and engineers while solving the design 
task. The sketches and notes however, acted as a reference to 
clarify some sections in the protocols. 

The purpose of segmenting is to break the transcribed 
verbal protocol text into units (or segments) that can be coded 
with a pre-defined coding scheme. Segmenting took place in 
two stages. In the first stage, larger units of analysis called 
think-aloud utterances were identified and segmented from 
each other. Think-aloud utterances are comprised of those 
words spoken aloud by a participant that were followed by 
some period of silence (Hartman, 1995). These periods of 
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silence or pausing had to have duration of five or more 
seconds. A total of 70 utterances were segmented (40 for the 
engineer and 30 for the engineering student). The think-aloud 
utterances were further segmented into sentences. 

Codes were provided for eleven predefined constructs. 
The constructs, their codes, and meanings are described in 
Table 1. Reliability coding was conducted by having one 
additional person code five pages of the first transcript. A 
reliability kappa coefficient of 0.86 was calculated which was 
well within the range recommended for interrater reliability 
(Miles & Huberman, 1994). 

Table 1 

Constructs, Codes and Definitions 


Construct 

Code 

Definition 

Analyzing Skill 

• ^identifying attribute and 
component 

IAC 

Make distinction between parts 
that together constitute a whole, 
e.g., The various parts of a nail 



gun. 

• ^identifying relationship 
and pattern 

IRP 

Articulate the interrelationships 
between components, e.g., 
rotating wheel and a gear 
mechanism. 

• Indentifying Error 

IE 

Detecting mistake in logic, 
procedure, calculation. 

Identifying an error in design 
e.g., recognizing a component 
won’t work because of a design 
flaw. 

Generating Skills 

• Inferring skills 

IS 

Going beyond available 
information to identify what 
reasonable may be true. 

• Predicting skills 
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PS 

Assessing the likelihood of an 
outcome based on prior 
knowledge of how things 
usually turned out. E.g., Shaft 
will wear because tolerance is 
too small. 

• Elaborating skills 

ES 

Adding detail explanations, 
examples or other relevant 
information from prior 
knowledge to improve 
understanding, e.g., using 
knowledge of typewriter to 

Engineering Design Concept 


know how a computer works. 

• Constraint 

• Predictive analysis 

C 

Specifications, criteria and 
conditions that constrains an 
engineering designing task, e.g., 

PA 

A device to clamp soft surfaces. 
Methods analyzing the 
functionality of a designed 
component, e.g., Calculating the 
maximum stress a device can 
withstand. 



• Optimization 

O 

Obtaining the best design based 
upon prioritized constraints, and 
criteria. 


Solution Space 

SP 

Generating solution 

Problem Space 

PS 

Framing the problem 


Results 


Research Questions One 

Table 2 illustrates that the student performed twice as 
much analysis as the expert. However there were differences 
between the student and the expert in where the analyses were 
primarily performed. The expert did most of his analysis (15) 
when using engineering science and mathematical fonnula. 
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The student, on the other hand, did most of his analysis (34) 
when he was resolving issues relating to the various constraints 
in the design problem. Both the student and the expert’s 
analysis mainly focused on the relationships between the 
component parts of their design. 

Table 2 

Analyzing Skills Frequency Table 


Analyzing skill of Expert 


Constraint 

Predictive 

Analysis 

Optimization 

TOTAL 

Identifying attribute and 
component 

5 

4 

2 

11 

Identifying relationship 
and pattern 

5 

8 

2 

15 

Identifying error 

0 

3 

3 

6 

TOTAL 

10 

15 

7 

32 

Analyzing skills of Novice 

Identifying attribute and 
component 

14 

4 

2 

20 

Identifying relationship 
and pattern 

14 

7 

4 

25 

Identifying error 

6 

6 

6 

18 

TOTAL 

34 

17 

12 

63 


Research Question Two 

Table 3 illustrates that the expert used almost twice as 
much generating skills as the student. Most of these skills were 
used primarily when he carried out analysis and optimization. 
In contrast the student did most of his generating when he 
performed analysis. The expert also spent more time making 
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inferences and elaboration about the quality of his solution 
while the student spent more time predicting how each 
component will function. 


Table 3 

Generating Skills Frequency Table 


Generating skills of Expert 


Constraint 

Predictive 

Analysis 

Optimization 

TOTAL 

Inferring 

2 

13 

12 

27 

Predicting 

2 

10 

6 

18 

Elaborating 

12 

6 

8 

26 

TOTAL 

7 

31 

26 

64 

Generating skills of Novice 

Inferring 

2 

7 

2 

11 

Predicting 

6 

8 

4 

18 

Elaborating 

2 

4 

4 

10 

TOTAL 

10 

19 

10 

39 


Research Question Three 

Verbatim reports from the transcripts of both 
participants’ “think-aloud” session highlighted the strategy 
used by each to solve the design challenge. The comments are 
preceded by a timestamp. 
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The Expert 

While the expert had over 25 years of experience, the 
type of task he was required to solve did not fall within the 
general type of problems that he was accustomed to solving. 
Immediately after reading the problem, the expert formed a 
mental image of the solution. He used this mental image to 
frame and plan the strategy he would use: 

(00:33) What I would probably do is I see a mental 
picture of that hold down device in my head. I would 
probably go with a design that was similar to that 
because I knew that was the design that has been 
proven to work so in a way. I'm not creating something 
from nothing. I've already got an idea what that would 
look like. 

The expert used his experience and knowledge of 
moments and forces to resolve positional and functional issues 
and made decisions about the relationship and structure of the 
component: 

(03:03) I know that from experience there should 
probably be a lower pivot point. Just guessing at this 
point in time, so I'm going to sketch a lever that comes 
up that's pivoted and then there should be another 
connection from this lever, the lever on my arm that I'm 
using to raise and lower it. So now what I got here is 
some rough idea of the linkage. I think would work and 
then of course to hold this down to a clamping device. 
I have to realize that my lever here needs to go over 
center so the item that you're holding down cannot be 
the force, because it would not pull back and release 
itself. For instance, if I pivot this point here and I pivot 
this point back to here. What it is going to do to the 
lever that I install here is that it is going to actually pull 
back and lift this thing, or is it going to bind or 
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whatever. I may have to have some spacers or 
something to bring it out around the other point and ah 
something like this. 

The expert strategized his approach from a system 
perspective. He seemed to identity the most critical element of 
the system, and then focused his cognitive effort into solving 
the function of those components to meet his personal goal and 
the solution criteria: 

(10:00) ...once I know my linkage is going to work 
and if I pull the lever, this thing is going to open up and 
whatnot like I want it to. Then I'll go to more of a 
design process where I actually may come up with, I 
guess, the second design of the parts that actually would 
work. 

He also evaluated and reflected on his progress. If 
there was conflict with his approach and solution he considered 
alternatives, but his motivation to achieve his goal impelled 
him to use his experience or knowledge of physical principles 
to resolve the conflict: 

(25:00) ...if I use these parameters I'm going to be able 
hold the part down or would the part just push back and 
just push these lever arms up? That I don't really know 
at this point in time. Should I start this thing all over 
again or is this going to be a good design? Should I go 
back to the paper or should I come up with an 
alternative holding device? 

The Novice 

The novice used more time (approximately 10 minutes) 
when compared to the expert (approximately 3 minutes) to 
analyze the problem, clarify elements of the specifications that 
he could not understand, and gather data, before he started 
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generating solutions. His solutions were primarily influenced 
by the specifications identified in the problem. 

(00.34) ...specifications are to hold three inches of 
material, and have eight inches of reach which is 
somewhat ambiguous. I'm interpreting that as meaning 
that it can extend eight inches from the material so that 
it can be gripped. To be able to solve this next part, to 
release the work piece quickly, easy to position, and 
easy to move to other work areas so it's not too overly 
bulky or too heavy ...the holding strength should be 
considered. Okay so now the holding strength would 
depend on what type of material that we're going to be 
holding. 

The solutions generated by the novice appeared initially 
to be limited to the concept of two moving parts. He then 
started generating solutions around the two moving parts using 
mental pictures and imposing constraints as he progressed. 

(03:23) ...first thing that I'm going to do, I guess for 
something like this we're going to have moving parts. 
So I'll be designing probably at least two parts ...would 
probably be the holding mechanism some sort of screw, 
that's what I'm interpreting ... going to be able to adjust 
because we want to be able to let it grip below three 
inches, so it needs to have some sort of adjustment. 

As the novice progressed, he used analogies that were 
case specific to decide on the structure of the component. This 
was evident at various points during his solution. He also spent 
time determining the relationship of one component with the 
other. As his solution progressed, it showed a component-by- 
component pattern of solution which can be compared to using 
bottom-up generation of solutions rather than a top-down: 

(06:33) ...some kind of gripping mechanism on the top 
of the lever like some claws which is what we see on 
monkey wrenches...I'm going to design something 
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similar for the bottom now except that this has to be 
able to interact with the screw to let it move up and 
down... am I thinking that this other part is actually 
going to be sliding up and down the long side of the 
part. 

Like the expert, the novice also used knowledge of 
moments and forces to evaluate and resolve positional and 
functional issues and also to make decisions about the structure 
and relationship of the components. He also performed 
evaluation at various stages of his solution. 

(18:50)...I try to make use of symmetry so that it fits 
directly in the center. The last thing I would want to do 
is create offset forces in moments that could really 
mean it introduces bending stresses and that's just not 
something we want... Albeit a quick release 
mechanism something like this might be a little bit 
dangerous to have, which is perhaps why there aren't a 
lot of them.... 

Figures 3 and 4 illustrate the cognitive strategies used 
by the expert and novice respectively, as they navigate 
cognitively between the problem space and the solution space. 
The time spent in the problem space was determined by 
statements made by the participants when framing the problem 
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Figure 3 

Network diagram for expert 
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Figure 4. 

Network diagram for engineering student 
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and gathering data about specifications and constraints, while 
time spent in the solution space was determined by statements 
made by the participants when generating solutions, 
performing analysis, and optimizing solutions. The network 
diagrams show that the novice spent more time in the problem 
space than the expert. It also shows that the expert framed his 
solution around his mental model, while the novice framed his 
solution around the problem specifications. The novice 
depended mainly on the information provided by the problem, 
thus proving that he used the problem-driven strategy. In 
contrast the expert used the solution-driven approach, spending 
most of his time generating solutions and little time defining 
the problem. 


Conclusions 

This exploratory study highlights certain differences in 
the way an expert and a novice engineer used their analyzing 
and generating skills while solving a fairly ill-structured design 
problem. The expert tends to use more inferences and 
elaboration when solving the design problem and the novice 
tend to use analysis that is focused on the functional 
relationship between the parts of the designed component. 
This difference might be attributable to the mental models or 
analogies that they generate. The mental representations used 
by the expert not only allowed him to go beyond merely 
predicting the perfonnance of his conceptualization, but to also 
improve his solution by adding additional details that the 
novice, who because of his limited experience, was unable to 
add. 

The novice behavior was associated with a “depth-first” 
approach to problem solving. This approach is characterized 
by sequentially identifying and exploring sub-solutions in 
detail. He approached the solution component-by-component, 
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focusing on the adjustable jaws, then the screw, locking device, 
and lever. The expert in this protocol study showed a “top- 
down” “breadth first” approach, which is consistent with what 
research confirms of experts in some knowledge domains. He 
quickly recognized the system requirements and then focused 
his efforts on designing what he perceived to be the most 
critical element of the system—the lever locking mechanism. 
His approach might also be attributable to the fact that experts 
are exposed to a large number of examples, problems, and 
solutions that occur in their domain. Therefore a key 
competency of the expert is the ability to stand back mentally 
from the specifics of the accumulated examples, and form a 
more abstract conceptualization pertinent to the solution 
(Cross, 2004). This allows him to focus on the critical 
elements of the problem, rather than on superfluous details. 

Both the expert and novice carried out frequent 
monitoring and evaluation of their strategy and solutions to 
identify errors. The novice however, evaluated against the 
specifications and constraints dictated in the question, while 
the expert evaluated against his perceived functional goals. 
The engineer used his initial mental models to guide his 
solution and displayed more complex inferring and elaborating 
skills. He also spent more time in the solution space. Lloyd 
and Scott (1994), reported that experienced designers used 
more generative reasoning, in contrast to deductive reasoning 
employed by less experienced designers. In addition, their 
protocol studies of experienced designers found that they were 
more solution focused. The expert’s approach clearly was 
solution-driven. He selected a feature of the problem-space to 
attend (naming), and from there explored the solution space 
(Schon, 1983). The expert spends more time within the 
solution-space, while the novice spends more time within the 
problem-space (see Figure 3 & 4). 
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Finally, both the expert and novice in this study used 
knowledge of mechanics to evaluate and also predict functions 
and positional locations, as well as to identify errors. This was 
consistent with findings which showed that one of the 
commonalities between designers was their implicit or explicit 
reliance on ‘first principles’ in both the origination and detailed 
development of their design solutions (Cross, 2002). 

Implications for Engineering and Technology Education 

The findings from this study would suggest that a 
proper grasp of systems concept is necessary to raise the 
problems solving ability of students to be reflective of experts. 
How components interrelate with each other and to the entire 
system, whether it be a simple or a complex system, are 
important for students to understand in order to increase their 
ability to generate conceptual solutions and solve functional 
issues. This will enable students to spend more time in the 
solution space, like experts, rather than in the problem space. 
Requiring students to solve design activities in the classroom 
that are different in surface features, but may have similar 
underlying operational concepts, would broaden their schema 
of problems that fall within the same category, and allow them 
to generate solutions more fluently. The ability to make 
inference and elaborate is a critical skill that distinguishes the 
expert from the novice in this study. Curriculums that integrate 
scientific enquiry skills in the engineering design process may 
be a step in the right direction to develop student’s ability to 
infer and elaborate. According to Crismond (2007, p. 27), 
“Students can develop their own guidelines based on tests they 
conduct by formulating design rules-of-thumb.” As they 
evaluate scientific findings in relationship to their engineering 
solutions, they will grow in their ability to identify and add 
missing details of a solution as experts are able to do. 
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As efforts to infuse engineering design in technology 
education continues, the presence of these thinking skills and 
cognitive strategies emphasize that they cannot be ignored in 
the instructional process. Clearly, more studies need to focus 
on the variety of thinking skills and cognitive strategies used in 
engineering design and the most effective instructional 
techniques to develop these skills and strategies. 
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